The present invention relates to computer graphics, and more particularly, to creating vector maps in a computer graphics processing pipeline.
In a computer graphics pipeline, three-dimensional objects are typically rendered by assembling a plurality of polygons in a desired shape. The polygons are conventionally triangles 100 (See Prior Art FIG. 1) composed of vertices which are defined by spatial coordinates, color values, and texture coordinates among other values. As this polygonal representation consists of a collection of flat facets of fixed size, a texture map 102 (See Prior Art FIG. 1) is often applied to the facets in order to provide greater detail. This texture map describes a pattern to be applied to the surface of each triangle to vary the color or other properties. The texture map itself is a matrix of values typically interpreted as color components, and the placement of values in the matrix describes a pattern which, when applied to the triangles, may vary the colors in accordance with the pattern. The texture map elements are accessed by a texture coordinate, and the values stored at each integral texture coordinate constitute one texel (texture element) within the texture map. The texture map is typically loaded as a file from disk or created and filled with values by the computer system central processor.
Texture maps may also be interpreted and used as vector maps, whereby the texture map values do not represent the colors of a pattern, but instead represent multi-dimensional vectors within a pattern. For example, rather than storing the surface color at each texel of a texture map, one could provide a texture map which holds a surface normal vector at each texel. This map then represents a geometric surface at a level of detail independent of the polygonal vertex geometry. A texture map of this type, referred to as a normal map 104 (See Prior Art FIG. 1), may then be used in lighting and shading calculations in the graphics pipeline, and the normal map provides a more efficient representation of surface detail than the polygonal vertex geometry.
In general, normal maps 104 (See Prior Art FIG. 1) are one formulation of vector maps. Other typical vector map formulations include displacement maps for environment-mapped bump mapping (EMBM), displacement maps for other texture perturbation techniques, and height maps 103 (See Prior Art FIG. 1). Displacement maps hold a vector used to perturb texture coordinates during rasterization, and height maps consist of a one-dimensional vector which represents the geometric height of a surface at each texel. The one possible differentiating feature of a vector map is that its values are not necessarily intended to be directly visible in pixels of the rendered scene. Rather than containing colors which are blended into pixels of the scene, vector maps contain values that are input to various graphics pipeline calculations. These calculations use the values to control aspects of rendering such as texture coordinate values, texture reads, surface lighting, and bump mapping. An example of this may be found with reference to: Moller, Tomas, and Haines, Eric, Real-Time Rendering, A K Peters, Ltd., 1999, p 136; which is incorporated herein by reference.
FIG. 1 thus illustrates typical data used in computer graphics rendering. Geometry (i.e. triangles 100) may be combined with a plurality of color textures of a texture map 102 and vector maps to produce a rendered image. Color textures represent the physical appearance of the object as seen by the human eye, while vector maps do not necessarily have perceptual counterparts in the physical world. Instead, vector maps contain mathematical or geometric information about the surface. Such information may include the height of the surface of the object at each texel as in a height map 103 and the normal vector to the surface at each texel as in a normal map 104, displayed here through the use of a color map to make the data perceptible.
Vector maps are utilized in a graphics hardware pipeline as input data to a variety of rendering calculations. One such operation may be that of per-pixel lighting as illustrated in Prior Art FIG. 2, whereby illumination values 204 are computed at each pixel of the rendered object 205. This calculation involves, for each pixel rendered, a dot-product 201 between incident light vectors 200 and the surface normal vectors 202. The vector map, in this case a surface vector normal map, is useful only in the context of the rendering calculation. The vector map values do not necessarily appear in the output, though they determine, via calculation, the appearance of the output.
Vector map values may be stored in a variety of numerical formats including, among other formats, unsigned integers, signed integers stored in a biased and scaled range, signed integers stored in twos-complement format, integers of various bit precision including 8, 16, and 32 bits, and floating point formats.
Vector map values are typically, but not necessarily, derived from geometric values. A vector map may be calculated from the gradient of a scalar map, as in the case of calculating a surface normal map from a height field, or it may be calculated as a collection of difference vectors between two or more collections of vectors. As vector maps may thus encode geometric features of a surface or affect rendering in ways more sophisticated than those of the standard diffuse texture mapping, it is desirable to animate their values when used in rendering. In this way, animation of the vector map data is analogous to animation of the values, such as surface height, from which the vector map is derived. Unfortunately, animation of the vector map in a meaningful fashion without the underlying data from which it is derived is not practical and in some cases is not possible. To animate the vector map, the practical solution is to animate the data from which it is derived and recalculate the vector map from the data.
Such calculations to create a vector map are involved, and in the prior art were performed by a software program executing on the computer system central processor. Prior Art FIG. 3 illustrates a prior solution to the problem of creating a vector map 306 from a height map 302. To create a normal map texel value 304 corresponding to each height map texel 300, an area of neighbors t0, t1, t2, and t3 surrounding the height map texel 300 are sampled and their values differenced. The differences produce two slope vectors dZ/dU and dZ/dV, the cross product of which is the three-dimensional surface normal vector. This vector may be stored as one texel of a texture map and the texture map transferred to the graphics pipeline. This method of creating the vector map may be performed on the CPU according to the prior art method of FIG. 3A.
Under this scheme as shown in FIG. 3A, the resulting vector map 306 must be transferred via a bus to the hardware graphics pipeline 312 for use in hardware-accelerated rendering. The high CPU cost of the calculations and bandwidth required to transfer the vector map to the graphics hardware severely restrict the use of this prior method with animated vector maps. Also, under the prior method, both the CPU and hardware graphics pipeline are prone to fall into an idle state at various times while waiting for the other to complete its operations. These idle states reduce the performance of the overall system. Under the prior method, the vector map and source data from which it is derived cannot be updated rapidly or efficiently for use in hardware accelerated rendering.
What is needed is a system in which vector map data may be calculated as needed in the graphics pipeline itself, independent of the CPU and without having to travel across the system bus from the CPU to the graphics pipeline. Vector map data generated in this way is available for immediate use in the hardware graphics pipeline. Such a system eliminates an interdependence between the CPU and graphics pipeline, allowing both to operate efficiently which in turn allows for rapid generation, animation, and use of vector map data in computer graphics rendering. Given the ability to create vector maps in a hardware graphics pipeline, the maps may then be used to modify themselves or their source data in further hardware graphics pipeline operations.
A system and method are provided for creating a vector map in a hardware graphics pipeline. Initially, one of a plurality of transforms is selected in a hardware graphics pipeline. Further, input is processed in order to generate a vector map utilizing the selected transform in the hardware graphics pipeline. Subsequently, a plurality of pixel color values is rendered utilizing the vector map.
In one embodiment, the input may include a height map. Further, the height map may be generated by the hardware graphics pipeline. As an option, the height map may be stored in the hardware graphics pipeline and may be updated by a central processing unit. In such case, the height map may be capable of being updated while saving bandwidth between the central processing unit and the hardware graphics pipeline.
In another embodiment, the input (i.e. height map) may be generated based on geometry information. Further, the input may be processed utilizing a rendering process of the hardware graphics pipeline. As an option, the vector map may be directly generated by processing input in the form of geometry information. Again, this may be carried out in the hardware graphics pipeline. In still another embodiment, the input may include basis values. Of course, the input may include any other input capable of being used to generate a vector map.
In still another embodiment, the vector map may include a bump map, a normal map, an environment-mapped bump mapping displacement map, and/or a displacement map. Optionally, the displacement map may include both positive and negative displacements. Further, the vector map may include a biased and scaled displacement map.
In the case where the input includes a height map, the height map may include a plurality of height map values that are each received in the form of a texel. In use, the processing may include sampling a plurality of neighboring texels surrounding a texel corresponding with each height map value in order to generate vectors. The sampling may include calculating a plurality of offset values associated with each height map value of the height map. Such offset values may each be associated with one of the neighboring texels. Further, the offset values may each be calculated by adding a vector value and a centering value. Still yet, the sampling may further include generating a plurality of vectors based on the offset values.
As an option, the aforementioned generated vectors may include gradient values. Moreover, the processing may include generating vector map values associated with the height map values based on the vectors. The processing may also include normalizing the vectors, and/or any desired filtering.
In still yet another embodiment, the processing of the vector map may occur in multiple passes. This repetition may be facilitated by way of a feedback loop. Such feedback loop may be biased by a gravity or other influence vector or a plurality of vector maps. Further, the processing may be repeated in a programmable manner. As an option, a vector map may be used to modify the height map of a subsequent pass. As an option, the vector map may be filtered before using the vector map to modify the height map. The foregoing operation may even be replicated, and the processing may be based on additional input.
In still another embodiment, the processing further includes writing the vector map values to a texture map. As an option, the rendering may occur immediately following the processing.
Optionally, the hardware graphics pipeline may include programmable pixel shader hardware. Further, the hardware graphics pipeline may include programmable vertex processing hardware. One example of an application of the foregoing technique involves the use of the vector map for per-pixel fluid rendering.
In another example of use, a first height map may be generated in a hardware graphics pipeline. Next, the first height map may be fed back in the hardware graphics pipeline. Thus, a second height map may be generated in the hardware graphics pipeline utilizing the first height map.
Thus, rather than relying on a central processing unit (CPU) to calculate a vector map from a height map or other data, the foregoing technique is designed for configuring the rendering operations of a hardware graphics pipeline such that the act of rendering calculates the vector map.
The input data from which the vector map is calculated may be stored in one or more textures in video memory. These textures may be sampled and combined in optionally programmable vertex processing and pixel processing hardware. The resulting output of the calculations may be rendered to a video memory surface (i.e. a texture) for later use as an input texture for lighting, reflection or any other type of 3-D rendering.
Thus, the present embodiments provide a technique for configuring a hardware graphics pipeline such that the operations of rendering calculate the desired vector map data from a variety of source data input to the graphics hardware. The source data includes one or more texture maps applied to polygonal geometric primitives. In the process of rendering, the pipeline configuration first creates and samples sufficient information to calculate the desired vector data. As rendering progresses, the vector data is then available for use in further calculations. It may be used, discarded, and/or written to graphics pipeline memory for later use, typically in the form of a vector map texture. This method creates vector map data on demand as needed by the hardware graphics pipeline. Given the ability to queue a host of rendering operations for efficient pipelined execution, this method allows the hardware graphics pipeline to create and utilize vector map data independent of the computer system central processor (CPU).
Vector map data created in this manner may be fed back into further rendering operations that in turn calculate additional vector map data. The method of this invention enables cascades and loops of the data through multiple rendering operations such that vector maps in the graphics hardware may perturb and modify themselves over time and in so doing, give rise to useful animations. In addition, the input data or rendering loops may be modified or perturbed by the host program executing on the CPU, and the configuration of the graphics pipeline calculates vector map values as required.
This method has the additional benefit of abstracting the expression of vector map calculations from the details of the actual underlying calculations being performed by the graphics pipeline. This is to say the graphics pipeline is configured and instructed to perform certain rendering operations that are guaranteed to yield appropriate vector map values. The details of how the graphics pipeline chooses to perform these calculations at each element, the degree of parallelism of the calculations, the time at which each element is processed, the order in which each element is processed, and the method of storage of the input and output data may vary per the desires of the user.